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DETAILED ACTION 

Claim Rejections - 35 USC §112 

1 . The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

2. Claims 1 ,4-1 3 are rejected under 35 U.S.C. 1 1 2, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. The current claim language calls for a "gaze 
projection of a position of the head through outer edges of the virtual window". It is not 
clear what the applicant is trying to say with this limitation. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1 966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 
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5. Claims 1,4-19, 21 and 46-57 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kang US Patent 6,009,210 and further in view of Kanade et al. US 
Patent 6,1 51 ,009. 

6. Kang teaches a method for processing interactive user control for a view of a 
scene displayed on a virtual window comprising: 

7. In regards to claims 1,14,46,47,52,53,57 

a. Identifying a head of a user that is to interact with the scene (Col 1 lines 
43-47 and Col 3 lines 29-32) 

b. Storing an initial frame of image data representing the head of the user 
(Col 3 lines 29-35) 

c. Tracking the identified head of the user during display of the scene, the 
tracking enabling detection of a change in position of the head of the user 
(Abstract) 

d. Tracking including identifying a search region within a frame of the user 
image data and comparing values within the search region to template values of 
the initial frame of the stored image data; adjusting a view frustum (which is 
being interpreted as the "view of the user into the virtual world") in accordance 
with the change in position of the head of the user (Col 2 lines 34-43 and Col 
3 lines 4-21) 

e. Repeating the identifying the search region, the comparing, and adjusting 
for successive frames of the scene, wherein the comparing is performed with the 
initial frame of the stored image data (Col 2 lines 34-43 and Col 9 lines 17-44) 
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f. A computing device and a display screen in communication with the 
computing device configured to display image data defined through a view- 
frustum (Fig 1) 

g. Tracking device is a camera (Fig 1) 

8. In regards to claim 15, a view frustum initially defined by a gaze projection set 
between outer edges of a virtual window and a virtual position of the head when the 
virtual position of the head is normal to a center point of the virtual window (Figs 1-3) 

9. In regards to claim 1 6 adjusting the view-frustum moves the virtual position of the 
head away from normal relative to the center point of the virtual window (Col 2 lines 34- 
43) 

1 0. In regards to claim 4,17, virtual position of the head being away from normal 
relative to the center point of the virtual window changes an angle of the gaze 
projection, the change in angle of the gaze projection displays a change in viewing 
angle of the scene provided by the video clip (Col 4 line 60- Col 5 line 49) 

11. In regards to claim 5, the change in viewing angle of the scene is a result of the 
detected movement of the head of the user to enable the interaction with the scene (Col 
4 line 60- Col 5 line 49) 

12. In regards to claim 1,6-7,14,46,49,50 Kang fail to specifically teach 

h. Adjusting a scale of the scene according to a change in a distance of the 
head of the user from a capture device 

i. Using a capture device having depth-capturing capability. 



Application/Control Number: 10/663,236 Page 5 

Art Unit: 3714 

1 3. Kanade teaches the use of a depth capturing camera for interaction with a view 
of a scene (abstract) 

14. One of ordinary skill in the art, at the time of the invention, would have been 
motivated to have a system that interacts with a view of a scene using depth capturing 
device to adjust a scale of the scene according to a change in a distance of the head of 
the user from the capture device. Using a depth camera, the system would be able to 

j. Obtain distance or depth information from the object being tracked 
(inherent of "depth camera) 

k. Determine any interaction such as occluding, shadowing, reflecting or 
colliding (Kanade abstract) 

I. Generate appropriate output based on said determination (Kanade 
abstract) 

15. In regards to limitations that further defined and claimed the definition of 
the view frustum defined by a gaze projection of a position of the head through 
outer edges of the virtual window when the position of the head is substantially 
normal to about a center point of the virtual window. Applicant should 
respectfully note that this is the known view projection associated with depth 
cameras. Depth cameras acquire and process recieved image data by performing 
a coordinate transformation on each recieved pixel. The depth information is 
processed by reading the objects current location in reference to the windows 
center point (i.e. the need for calibration and storing initial image) and 
transforming the received location to an (x,y,z) coordinates. Where x and y 
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corresponds to a position that relates to a position on the display screen and z 
corresponds to the distance from the position to camera. Hence the projection. 
A constant focus on the object has to be maintained throughout the process for 
the system to be able to maintain the correct coordinates. 

16. Wherein the adjusted view frustum defined by an updated gaze projection 
of the changed position of the head through the outer edges of the virtual window 
((Col 2 lines 34-43 and Col 3 lines 4-21), examiner interprets this limitation as, 'as the 
user moves his/her head, the view frustum is changed to reflect said change'). In 
regards to laterally adjusting the view frustum in a direction opposite to the 
change in position of the head of the user (see cols 3-4 of Kang), applicant should 
respectfully note that the location of the user's head and the location of the view 
frustum are opposite each other (i.e. the user is facing the view). Thus, they have 
different orientation (i.e. the user's left is the view frustum's right). Furthermore, 
the examiner interprets this limitation as ' as the user moves his/her head, the 
view frustum is changed to reflect said change '. For example, if the user moves 
his/her head to the left, the vertex of rotational angle (i.e. the viewpoint around 
the viewing axis) will need to shift to the right (in respect of the user) to show the 
scene on the left of the user. Similar to if a person wants to move his/her head to 
see what is on the left, the back of the person's head has to move to the right 
while the persons view moves to the left to see what is on the left 

17. In regards to claim 8, the initial frame of image data is marker-less (Fig 1) 
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18. In regards to claim 9, the initial frame of data is maintained throughout the video 
clip (Fig 1) 

1 9. In regards to claim 1 0,48, the video clip is of a video game. Although Kang 
teaches the method to be generally associated with virtual environment in computer 
systems, Kang further teaches that video games are well known in the art as virtual 
environment in a computer system (Col 1 lines 26-40) 

20. In regards to claim 1 1 , the interaction with the scene by tracking movement of the 
head of the user is independent of user hand-held controls for interacting with the video 
game (Col 1 lines 5-9) 

21 . In regards to claim 12, the method operation of tracking the identified head of the 
user during display of the video clip includes tracking a facial portion of the head and 
matching gray scale image data associated with the facial portion to image associated 
with a template of the facial portion (Col 9 lines 20-33) 

22. In regards to claim 1 3,18-1 9,50, the method of adjusting a view frustum in 
accordance with the change in position of the head of the user includes identifying a 
point of interest of the scene of the video clip and modifying the view-frustum so that the 
point of interest appears at a constant position when displayed in successive video clips 
(Kang Claim 6). The Examiner should point out that each frame in Kang's disclosure 
has to be scanned before the scene transformation could occur. 

23. In regards to claim 21 ,49-50, the method operation of translating a view frustum 
in accordance with the change in position of the head of the user includes rotating the 
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view-frustum about the head of a user according to the change in position of the head of 
the user (Kang Claim 5). 

Response to Arguments 

24. Applicant's arguments filed 7/1 6/07 with respect to claims 1 -26,28-44 and 46-57 
have been considered but are moot in view of the new ground(s) of rejection. 

Response to Arguments 

25. Applicant's arguments filed 12/04/07 have been fully considered but they are not 
persuasive. 

26. On page 20, applicant argues, "Further, Kang would not adjust a scale of the 
scene according to a change in a distance of the head of the user from a capture device 
as specified in claims I, 14, 22, 28, 33, 39, 46 and 52, or include a camera having depth 
capability as specified in claims 7 and 31, since the change in relative object depth 
compared to the distance to the camera must be small under the affine model (see 
column 5 lines 58-63). For the affine model to work, it is required that the surface 
appear planar. In order for the face to appear planar, the distance from the user to the 
camera must be relatively large. The use of a depth camera would render moot the 
entire affine model principles for which Kang is constructed" 

27. The examiner respectfully disagrees. Kang's invention is directed to 2-D affine 
"transformation". Kang teaches that the technique is suitable for planar surfaces with 
an affine camera model especially when the user is far from the camera. Therefore, a 
depth camera would not render moot Kang's affine model technique. Please see Col 
6:44-60 of Kang. 
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28. On page 21 , applicant argues, "The Examiner indicates that the teachings of 
Kanade would motivate one skilled in the art to interact with a view of a scene using 
depth capturing to adjust a scale of the scene according to a change in distance of the 
head of the user. Although Kanade teaches the identification of depth, the depth is only 
needed to capture the user, who needs to stand in from of a backdrop. These teachings 
do not teach to adjust the scale of the scene. In fact, the position of the user's head is in 
no way tied to any adjustment in a scene. The only distance tracking done is to identify 
where the user is located, so that objects can be positioned in and around the scene. 
As a further distinction, the "scene", as claimed is different from the area captured by 
Kanade. Specifically, in the claimed invention, the view-frustum is adjusted in 
accordance with the change in position of the head of the user and the adjusting of a 
scale of the scene takes placed according to a change in a distance of the head of the 
user from a capture device. In Kanade, the user is "in the scene". That is, the user is 
captured, and then injected into the scene. What the user does in the scene in no way 
performs adjustments. Indeed, if the user moves his head, the scene is not adjusted." 

29. However, Kanade is not being relied upon for the teaching of adjusting the scale 
of the scene. As shown above, Kang teaches adjusting the scale of the scene based on 
user location information received from a camera. Kanade is being used to show that 
using depth capturing image to capture depth information of a user during a gaming 
session is old in the art (Kanade abstract). 
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Response to Arguments 

30. Applicant's amendments/arguments filed 4/28/08 have been fully considered. 
Please see paragraph 13 above which addresses the newly added limitations. 

Response to Arguments 

31 . Applicant's arguments filed 1 1/14/08 have been fully considered but they are not 
persuasive. 

32. On pages 11-12, applicant argues, "The first method taught is "to directly use the 
pose parameters to determine the absolute position and orientation of the viewpoint" 
(Col. 8, lines 27-29). In other words, movement of a user's head in a given direction 
produces movement of the view of the virtual reality environment in the same direction. 
The second method taught applies the same directional system, but incorporates 
incremental control to "indicate continuous movement within the virtual reality 
environment" (Col. 8, lines 32-40). In contrast to Kang, Applicants' claim 1 is directed 
towards a view of a scene comprising a view-frustum defined with reference to a gaze 
projection of a position of the head of the user through the outer edges of a virtual 
window. The volumetric virtual space beyond the virtual window which is delimited by 
the gaze projection through the outer edges defines the view-frustum. And as the user's 
head moves, the view-frustum is adjusted accordingly, such that the gaze projection 
through the virtual window is updated in light of the changed position of the user's head 
to determine the new view-frustum. Thus, in contrast to the direct use of pose 
parameters to determine viewpoint disclosed by Kang, Applicants' claim 1 utilizes an 
indirect method wherein the view of the scene is defined with reference to a virtual 
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window. Hence, a movement of a user's head in a lateral direction will have the 
perceptual effect of shifting the view of the scene in the opposite direction. Kang does 
not disclose a view-frustum defined with reference to a virtual window, nor the 
adjustment of such a view-frustum in the manner claimed." 

33. The examiner respectfully disagrees. In Kang's invention, the player is looking at 
the virtual reality through an angled view (i.e. the angle about the viewing axis). As the 
player's head changes in position, the vertex of said angle will need to move in an 
opposite direction in order to move the scene in the same direction that the head is 
moving. This is not different from applicants claimed invention. 

Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See M PEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to EMMANUEL OMOTOSHO whose telephone number is 
(571)272-3106. The examiner can normally be reached on m-f 10-6. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Peter Vo can be reached on (571 ) 272-4690. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

EO 



/Ronald Laneau/ 
Primary Examiner, Art Unit 3714 
02/27/09 



